Feature Engineering and Feature Scaling

Understanding Feature Engineering in Model Building

Feature engineering is crucial for enhancing the predictive power of models by transforming raw data into meaningful inputs. Key aspects include:

Encoding Techniques:
- Ordinal Encoding: Converts categorical features like credit ratings into numerical form, preserving order.
- Label Encoding: Applies to target variables, like buy/sell/hold decisions.
- Cyclical Encoding: Uses trigonometric functions for cyclic features, such as months.
- One-Hot Encoding: Suitable for non-ordered categorical data like ETFs/stocks, though care is needed to avoid multi-collinearity.
Discretizing and Polynomial Features:
- Converts numerical data to categorical by defining "bins" (e.g., price ranges).
- Polynomial features involve higher-degree terms for capturing non-linear relationships.
Interaction and Combined Features:
- Combining features (e.g., BMI from weight and height) can provide more insights.
Feature Scaling:
- Scaling aligns feature ranges, aiding model learning, though not always necessary, e.g. for tree-based models.

Prioritize the quality of features over quantity to maintain efficiency and clarity in model training and interpretation.

SOLUTION:

Using one-hot encoding for categorical variables can help preserve information while allowing models to interpret these variables correctly.
Discretizing is when we create discrete values from a continuous-valued feature.
Feature scaling is not necessary when using tree-based models, as they are invariant to feature scaling.